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THE QUALITY OF GRADUATE STUDIES: 

PENNSYLVANIA AND SELECTED STATES 

INTRODUCTION 

It Is a minor paradox of higher education that graduate 
study, acknowledged to be the most costly and complex phase of 
university activity, should have the least attention* In- 
stitutions vary widely in the amount of self-inquiry they direct 
toward graduate work, and information collected over a wider 
area by standard methods has been difficult to come by. Several 
national studies conducted recently under the auspices of the 
American Council on Education however, offer a useful means of 
making local and regional comparisons • ^ When the data are re- 
arranged into regional or local subsets, the kind of relation* 
ships that are revealed give clues about the effects of past 
policy and future planning, clues that are not attainable by 
other means. 

This study collects and organizes the basic data from the 
Roose-Andersen evaluation of graduate faculty to give comparisons 

^Kenneth D. Roose, and Charles J. Andersen, A Robing of 
Graduate Programs (Washington, D.C. : American Council on Education 

1970) I and John A. Granger, The American Graduate Student: A 

Normative Description (Washington, D.C*: AGE Research Reports) 

6 (5) 1971* 
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that have special significance for Pennsylvania* The collected 
information for the Commonwealth is compared with similar data 
for New York* Illinois, Michigan, Ohio, and California and for 
the top ranked fifty schools In the study. The nature of the 
relationships between enrollment and ranking, the number of 
fields and ranking, and resident students enrolled and ranking 
make possible some rather clear observations about graduate 
education in the state. 

GRADUATE EVALUATION: 

A LOOK AT PRECEDENTS 

In 1861, a precedent was set in American higher education 
for in that year Yale awarded the first Ph.D.’s in American 
history* It is fair to assume that the ink was hardly dry on 
the parchment before questions arose concerning the quality of 
that small doctoral program at Yale* "By 1876, the year that 
Johns Hopkins dedicated itself to the development of the Ph.D* , 
the precedent set by Yale was being followed in twenty-five 
institutions, which that year awarded a total of forty Ph.D. 
degrees." By this time, the American penchant for competitive^ 
ness must certainly have spawned a desire to know which program 

2 

F» Rudolph, The American College and University (New York; 
Vintage Books, 1962), p . 335* 
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was the best and why* Such judgments were no doubt made, but 
they appear to have been made individually and with varying 
degrees of individual knowledge* 

By the turn of the century however, it became obvious that 
the task of evaluating or grouping schools offering studies 
leading to the doctorate needed to be approached with more rigor* 
It is not surprising therefore that in the year 1900, fourteen 
institutions, which together accounted for 88 percent of all 
doctorates awarded at that time, came to form the Association of 
American Universities* While the AAU never set about formally 
to evaluate institutional programs, it has served a function 
which might be called evaluation and certification by regulating 
admission to its ranks. Proposals to make more specific in^ 
quiries have been suggested, but not adopted — albeit narrowly 

o 

so. Nonetheless, membership in AAU has served as an indicator 
of academic quality to many, including German institutions 
which at the time of AAU 1 s formation were attracting numerous 
American scholars* Membership has grown from the original 14 
to the present number of 46* To see clearly that membership is 
still an evaluative measure, one need only note that those 46 
members stand in an exceptional condition when compared with the 

^G.O. Arlt, "Purifying the Pierian Spring," The Graduate 
Journal VIII (1971): 267-276* 
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Council of Graduate School’s current membership of 300# Member- 
ship in CGS is open to institutions that have granted thirty 
graduate degrees , A*M. * M.3., or Ph.D. , in at least three fields 
over a three-year period# 

In 1925 s Raymond Hughes undertook a modest but pioneering 

study of the quality of graduate training which was reported to 

4 

the annual meeting of the American Council on Education# This 
first formal national ranking was followed in 1934 by another 
study performed by Hughes# The place of formal evaluation of 
graduate departments by peer judgment had now been established. 

Hayward Keniston was commissioned in 1957 to evaluate 
graduate programs in the arts and sciences at the University of 
Pennsylvania. In order to compare the quality of programs in 
twenty-eight fields at Penn, Keniston collected and analyzed 
evaluations of that institution along with the other members of 
the AAU, which than numbered twenty-five. 

In addition, various professional organizations have under- 
taken evaluation studies of graduate programs in their specific 

^D.A, Robertson, ed. , Americcm Universities and Colleges 
(New Yorks Scribner, 1928), pp. 161-163. 

Keniston, Graduate Study and Research in the Arts and 
Sciences at the University of Pennsylvania (Philadelphia: Uni- 

versity of Pennsylvania Press, 1959). 
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disciplines. Such research has been done in history, English, 
Ame rican literature, physics, sociology, psychology, education, 
and certain other fields. 

During the 1964~1965 academic year, Allan M. Cartter, then 
vice-president of the American Council on Education introduced 
a new evaluation of the quality of graduate programs. 6 Although 
there is obviously no perfect way of making such evaluations , 
many believe the Cartter method to be the most thorough yet 
devised. At first glance it might appear that such objective 
yardsticks as size and "quality" of faculty, nature of facilities, 
type and quantity of funding, etc. might yield a reasonable 
indication of relative quality among institutions. However, 
these physical attributes may or may not directly affect the 
quality of the resulting educational experience. Cartter chose 
to use the tool of peer evaluation by soliciting the opinion of 
scholars in regard to their view of quality departments in their 
field. Such a process is consistent with the use of professional 
peer judgment as it exists in the medical, legal, and engineering 
professions. 

It was specifically suggested by the study advisory com- 
mittee that the 1964 study be replicated within five years "to 



6 Allan Cartter, An Assessment of Quality in Graduate "Programs 
(Washington, D.C.: American Council on Education, 1966). 



avoid ’fixing' refutations when in fact the academic scene is 
changing constantly," Following through on this recommendation , 
a replication with certain modifications was conducted in 1969 
by Kenneth D. Roose and Charles J. Andersen. 

As with the Cart ter study before it, the Roose- Andersen 
report triggered criticism in regard to rationale and methodology 
As Logan Wilson, president of ACE pointed out, "In general, the 
unhappy critics are those whose institutions did not stand up 
in the ratings . In bearing out this point, Dutka indicates 
great dissatisfaction with the report at Columbia, the institu- 
tion which appears to exhibit the greatest slippage between 
1964 and 1969. She notes, "Most professors quickly point out 
the inability of outsiders to accurately judge the effectiveness 
of a program, the vagueness of the wording on the questionnaire, 
the random character of the sample, and the built-in time 

i tv 3 

lag. . . 

There have been concerns which are not tied to a particular 
institution's individual woes. Jacobson notes the following 
points which arose at an AGE news briefing to discuss the reports 

^R.L. Jacobson, "Ratings of Graduate Departments Raise Ques- 
tions About Who's 'Best'," Chronicle of Higher Education V 
(January 11, 1971): 9 . 

®E« Dutka, "Poor Marks for Columbia Graduate School," Change 
3 (1971): 33-34. 
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The selection of respondents was made on certain 
criteria more than ten years old, thereby creating 
a bias against newer programs. 

2) Some evaluators may have used the catchall M not 
sufficient information" as a kind way of express- 
ing low esteem. 

3) The disciplines studied excluded a number of fields, 
including education, agriculture, and medicine. 

4) All evaluations were one and one-half years old 
when reported. 

5) Some fields had more judges than others. 

While the method of peer evaluation may be subjective, a more 

objective assessment of the outputs of higher education has not 

9 

yet been fully operationalized but is under investigation ■ In 
any case, decisions such as what students apply, what faculty are 
attracted, and what grants are awarded are probably made largely 
on the basis of subjective judgments rather than truly objective 
criteria. Reputation, no matter how imperfect or inaccurate, is 
a factor to be recognized. 

The initial Cartter report and the Roose-Andersen study of 
graduate quality dealt with two components — rated quality of 
graduate faculty and rated quality of doctoral programs. The 
present study deals exclusively with the first component, an 
emphasis that has been traditional for Inter-university compari- 



sons * 



Lawrence, G. Wea thereby, and V.W. Patterson, eds.. 
Outputs of Higher Eduoation^VIlCBE, July 1970. 
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.AMERICAN COUNCIL ON EDUCATION STUDIES 



The Cartter study reported strengths of leading individual 
institutions by general areas of study* This was as close as 
the study came to making overall judgments. Any evaluation based 
on a combination of rankings was eliminated from the 1969 data. 
Roose and Andersen state, "In this new survey, we have tried 
to deemphasize the pecking order relationships inherent in most 
scoring systems, for it is not our purpose to bolster or de- 
flate egos. We have, therefore, not presented scores for in- 
dividual institutions* 

However, if ACE was not willing to publish comparative 
rankings of institutions, others were quite ready to draw in- 
ferences that depended upon an aggregation of the departmental 
data, Jacobson reports, "The councils report scarcely had been 
made public when a number of universities sent out news releases 
of their own, mainly to call attention to survey results that 



were favorable to them. At least one institution used its own 
statistical weighting to give itself a higher comparative 
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Roose and Andersen, A Rating of Graduate Programs , p. 2. 
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standing than It had attained in the list of those most often in 
the top five. 

Raymond Ewell has developed a simple method for constructing 

composite scores for institutions. The approach, which was 

first used in reference to the Cartter study, is described 
12 

elsewhere. Basically, the technique is as follows. In- 
stitutions are awarded points in accordance with their ranks 
in individual fields. For each field, a university gets the 
points indicated below. The composite score for an institution 
is the sum of the points for all fields in which it was rated. 

DISTINGUISHED/ STRONG- - - - - -Inverse rank + 14 points* 
GOOD- ------------ -10 points 

ADEQUATE PLUS 5 points 

*BY USING THIS METHOD, WEIGHT IS GIVEN TO FIELDS WHICH 
HAVE THE GREATEST NUMBER OF DISTINGUISHED/STRQNG 
ENTRIES. THE CONSTANT OF 14 ASSURES THAT EVEN THE 
LOWEST IN THIS CATEGORY GET FIVE MORE POINTS THAN THOSE 
RATED GOOD. 

^~R.L. Jacobson, "Ratings of Graduate Departments," 

p. 9. 

12 

r. Ewell , n A Quantified Summary of the American Council 
on Education Report 'An Assessment of Quality in Graduate 
Education' [Cartter Study] 11 mimeographed (SUNY at Buffalo; 

Office of the Vice-President for Research, December, 1967), 




9 



±z 



Ewell notes in his report, "I checked this rating system with 
Dr, Cart ter, and he felt that this was the best of several systems 

which had been proposed to convert his disciplinary data into 

13 

composite ratings for inter-university comparisons. " 

When the departmental evaluations gathered by the Roose- 
Andersen study are weighted and summed by Ewell's method we 
have a composite score for each institution. Such a conversion 

was tabulated by J. William Johnston, and his summary provided 

» 

the basic institutional scores cited in the paragraphs and 

14 

tables that follow# The relation of institutional character- 
istics and department ratings has been the object of some 
speculation but not of definitive study#^ 



^Ibid. , p# 2. 

Johnson, "A Composite Ranking of Institutional 
Graduate Disciplines Based on the American Council on Education 
Report f A Rating of Graduate Programs' [Roose-Andersen study]" 
mimeographed (University Parks The Pennsylvania State 
University, September 1, 1971). 

15 R. Smith and F.E# Fiedler, The Measurement of Scholarly 
Work in Academic Institutions ; Technical Report No. 70-2 
(Seattle, Washington: University of Washington, February, 

1970). 
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THE AIM AND METHOD OF THE STUDY 



In the present study certain objective information about 
institutions is compared with the aggregated Ewell ratings. The 
central focus is on Pennsylvania institutions. They are compared 
with those of Ohio* Michigan* New York, Massachusetts* Illinois* 
and California* This frame of reference allows some comparison 
of the collective rankings for Pennsylvania with states that 
are similar in population: Ohio, Illinois, and Michigan, and 

with states that have a somewhat similar pattern of public and 
private institutions: Ohio, Massachusetts, and New York* It 

also permits a contrast of Pennsylvania with states that are 
heavily committed to public higher education: California, 

Michigan, and Illinois, Pennsylvania merits this distinctive 
set of comparisons because of its unusual pattern off public 
support. For some years now it has been the practice to provide 
Commonwealth funds to three classes of institutions that are known 
as "state-owned," ’'state-related , 11 and private "state-aided*" 

From total higher education appropriations of $261 million in 
1970-71, the fourteen "state-owned" colleges have been financed 
by the state in the amount of $73 million. A larger segment 
of the state funds for higher education has been selectively 
distributed — some would say scattered— among the "state -related" 
institutions ($148 million) and the "state-aided" institutions 
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($23 million) . The Roose-Andersen data in an aggregated form 
offer a means of commenting on this practice of distributing 
support among a group of quasi-public and semi-public in- 
stitutions. 

Another set of more general comparisons is also reported 
in this study. The rankings which originate with professorial 
judgments are correlated with other institutional attributes 
to see whether there are any marked associations between the 
gross features of an institution — total enrollment, graduate 
enrollment, for example — and the collective score or rankings. 

Roose has recently studied the characteristics of the 50 
top-rated institutions, drawn from the 130 whose programs were 
rated in 1969. He feels that analysis of the top 50 is 
especially critical in a period characterized by budgetary 
restraint and limited expansion. For this reason, the present 
study considers characteristics of the top 50 as well as of the 
selected states. In all comparisons, each institution is 
described in terms of the following characteristics: 

1. Rank-order of institution by composite score 
2 * Ewell composite score 
3» Number of fields rated 

4. Total enrollment 

5. Graduate enrollment 




15 



12 



6. Percent of total enrollment accounted for by 
graduate enrollment 

7. Type of control, public or private 

RESULTS 



Preliminary Comments 

Three qualifications need to be stated at the outset of this 
presentation* First, the present study is essentially a descrip- 
tive one* It simply introduces a comparison of relationships 
that were not included in the original Roose-Andersen analysis 
but which may show interesting relationships. The features 
portrayed are characteristics of the educational landscape rather 
than origins of institutional excellence. Second, the effect 
of a top-rated department on the judgments given other depart- 
ments within that institution cannot be accurately gauged. ^ 
However, it would seem reasonable to hypothesize that a univer- 
sity is more than the sum of its parts and thus the effect of 
a top department would be more than merely additive. That is, 
if the present study errs slightly in measuring overall in- 
stitutional rating, it does so on the side of underplaying the 
total quality of individual institutions - 

^Smith and Fiedler, MeaBurement of Scholarly Work, 
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Third, it should be kept firmly in mind that the ratings are 
of graduate programs* What direct and indirect effects such quality 
has on the undergraduate education, research, and service activities 
of particular institutions is not only difficult to measure but 
highly idiosyncratic to the local campus setting. Spin-offs from 
graduate quality are, no doubt, plentiful. They are not, however, 
within the scope of this paper. 

The Top Fifty 

Table 1 above displays the institutions whose composite 
scores place them in the top fifty in the nation* The table 
also indicates number of fields rated, the various indexes of 
enrollments, and type of control for each institution. This 
particular collection of data allows us to examine the relation- 
ship between a ranking based upon peer judgment and several 
attributes of an institution. Product moment correlations were 
computed by Pearson’s method for the Intercorrelation of all 
variables* For convenience, a level of probability of ^ .05 
is taken as significant. 

We turn first to the relationship between composite score 
and the number of fields rated. Considering the fact that there 
is a prescreening which determines whether a field at a given 
institution is even included in the evaluation survey, one 
might hypothesize that the more fields an institution has rated, 
the better its position might be. This supposition is supported 
by the possibility that excellence in one field, mathematics for 
example, might have a ,! halo n effect upon judgments made of other 

14 
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TABLE 1 



TOP 50 INSTITUTIONS: RANKED BY COlffGSITE SCORE 



Institution 


Rank 


Composite 

Score 


Fields 

Rated 


Total 

Enroll. 


Graduate 
Enroll * 


% Total In 
Graduate 
Enroll. 


Control 

Fub=F 

Frivol 


Doctorates 

Awarded 

1946-66 


U. C . - Berkeley 


1 


1156 


35 


28863 


9144 


31. 7 


P 


32 38 


Harvard 


2 


1095 


34 


19136 


6268 


46.1 


I 


2433 


Michigan 


3 


964 


36 


37283 


10328 


27. 7 


P 


2603 


Stanford 


4 


945 


32 


11556 i 


5326 


46.1 


Z 


1935 


Yale 


5 


919 


34 


8665 


3035 


35. 0 


I 


1800 


Wisconsin 


6 


912 


36 


32000 


8782 


26.6 


P 


2730 


Chicago 


7 


853 


31 


10464 


6041 


57.7 


I 


2079 


Princeton 


8 


844 


32 


4756 


1478 


31.1 


I 


1721 


Illinois 


9 


819 


36 


44806 


9 387 


21.0 


P 


39 33 


Cornell 


10 


790 


33 


14102 


3870 


27.4 


1 


200 7 


u.c.l.a. 


11 


788 


34 


29070 


8759 


30.1 


P 


1005 


Columbia 


12 


734 


34 


17459 


6528 


37.3 


I 


2501 


U of Washington 


13 


662 


34 


30357 


6560 


21.6 


P 


909 


U of Penn 


14 


642 


32 


19417 


6606 


34.0 


I 


1044 


MIT 


15 


642 


20 


7730 


3390 


43.9 


1 


4 354 


Minnesota 


16 


611 


33 


58304 


845 7 


14.5 


P 


1470 


Johns Hopkins 


17 


60S 


26 


11278 


3072 


27.2 


I 


1385 


Indiana 


18 


579 


32 


47806 


11009 


23.0 


P 


675 


Texas 


19 


576 


32 


30628 


5538 


18.0 


P 


1229 


Cal Tech 


20 


513 


16 


1520 


7 74 


50.9 


I 


1714 


Northwestern 


21 


510 


32 


17239 


4084 


23.7 


I 


1123 


Brown 


22 


399 


24 


5042 


1528 


30.3 


I 


675 


Duke 


23 


398 


23 


7552 


1622 


21.5 


I 


481 


Purdue 


24 


397 


21 


34263 


5201 


15.2 


P 


2197 


North Carolina 


25 


390 


25 


15601 


3483 


22.3 


P 


592 


Michigan State 


26 


389 


30 


38758 


10439 


26.9 


P 


617 


NYU 


27 


363 


28 


34582 


14602 


42,2 


I 


1324 


Ohio State 


28 


352 


32 


388 34 


7602 


19.6 


P 


2215 


Washington U 


29 


344 


25 


11908 


2548 


21.4 


1 


454 


U of Rochester 


30 


330 


24 


8423 


2043 


24.3 


I 


548 


Iowa 


31 


300 


29 


18659 


4777 


25.6 


P 


1061 


Rockefelier U 


32 


295 


10 


138 


128 


92.8 


I 


4 


Brandeis 


33 


294 


15 


2707 


744 


27.5 


I 


71 


Case Western Res 


34 


291 


21 


10927 


3606 


33.0 


X 


792 


Penn State 


35 


288 


28 


33742 


5009 


14.9 


p 


1379 


Oregon 


36 


282 


24 


13980 


3530 


25.3 


p 


149 


U.C. - Davis 


3? 


281 


16 


10161 


2367 


23.3 


p 


100 


Colorado 


38 


259 


28 


18280 


4195 


23.0 


F 


589 


U of Pittsburgh 


39 


238 


26 


22067 


7665 


34.7 


P 


819’ 


U.C. - San Diego 


40 


236 


10 


30 70 


1148 


37.4 


P 


no data 


SUNY Buffalo 


41 


230 


24 


19113 


4781 


25.0 


P 


176 


Kansas 


42 


223 


26 


15791 


3198 


20.3 


P 


538 


Iowa State 


43 


209 


18 


16925 


2696 


15,9 


P 


1498 


Rice 


44 


209 


19 


2830 


816 


28.8 


I 


437 


Southern Cal 


45 


206 


23 


18692 


6223 


33.3 


1 


261 


Syracuse U 


46 


192 


26 


20254 


4382 


21.6 


I 


401 


Rutgers 


47 


187 


21 


15142 


5767 


38.1 


P 


459 


Vanderbilt 


48 


186 


21 


5558 


1198 


21.6 


I 


226 


Virginia 


49 


185 


18 


18379 


6861 


37,3 


P 


538 


Carnegie Mellon 


50 


168 


9 


5228 


1406 


26.9 


I 


974 



Sources: Rank, Composite Score (Johnston) 

Number of Fields Rated (Roose-Andersen) 

Control, Enrollment — a§ of 1968-69 — (National Center for Educational Statistics) 
Doctorates Awarded — (Doctorate Recipients from U-S. — N.A.S.) 
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disciplines; physics , astronomy, and engineering, for example. A 
significant relationship of r - .72 was found between the composite 
score and the number of fields rated. This suggests that an In- 
stitution striving to improve its position in these national rank- 
ings might be wiser to consider a strategy for general improvement 
in a number of fields rather than a concentration upon one or two 
"star" departments. 

It might also be hypothesized that the ratings might be im- 
proved by having more students enrolled since alumni might then 
constitute a greater share of the professionals making the judg^ 
ment. When enrollment of the institution was compared with the 
composite score, a correlation of .26 was generated. This degree 
of association does not support the hypothesized notion, but 
perhaps it is only the graduate students who should be counted 
since the rating deals with graduate activity. When graduate 
enrollment was compared with composite score, a slightly in- 
creased correlation coefficient was generated, r “ .34, but it is 
much too low to support any idea about a significant influence of 
one factor upon the other. Viewed through the eyes of an in- 
stitution that aspires to improve its ranking, these data suggest 
that merely undertaking a program for growth in the number of 
graduate students will not materially aid quality# 

Enrollment figures, however, are momentary and may or ma 
not accurately reflect a university's production of scholars 
over an extended period of time. When we consider the number 
of doctorates which each institution has awarded in the 
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twenty-year period ending in 1966, it is found that this figure 
correlates well with composite score, r ® ,72, significant, 

A larger ntsnber of professionals having graduated from a 
particular institution would appear to have a favorable effect 
on ratings in at least two ways. First, a greater number of 
raters will have experienced a direct exposure to the institution's 
programs. Second, more raters will have indirect knowledge of 
the institution's program through contact and interaction with 
colleagues who have attended that institution. 

In terms of relative quality as reflected in the composite 
scores of the top fifty there is little difference between the 
public and private groups. Private institutions show a slightly 
higher mean score, 510 vs. 470, but this is not significant. 

When each of the groups is examined by correlating scores with 
enrollment, some interesting distinctions appear. The public 
institutions tend to have significantly larger enrollments, as 
one might expect, with graduate students constituting about 
25 percent of the total. Private institutions with smaller 
average enrollments have a larger share of their students in 
graduate study, 35 percent. Among the public institutions there 
is a significant correlation between total enrollment and composite 
score, r ** .55, and a similar relationship between graduate 
enrollment and composite score, r * .68. This condition does not 
appear among the private institutions. 
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Focusing on Pennsylvania 



The next phase of comparison considers the Pennsylvania 
institutions which fall into the total group of rated institutions 
consisting of 130 universities nationwide* Because our concern 
is for the Commonwealth as a whole, all data for institutions 
in the Commonwealth has been grouped. Three Pennsylvania in- 
stitutions fall outside the top 50, but within the total 130 
(Bryn Mawr, Lehigh, and Temple), while four (Penn, Penn State, 

Pitt, and Carnegie-Mellon) are within the upper group. 



Table 2 A 



TABLE 2 

BASIC DATA FOR 

SELECTED INSTITUTIONS IN SEVEN STATES 
Pennsylvania 



Institution 



Rank 



Composite 

Score 



D 

Fields 

Rated 



E 

Total 
Enroll , 



F 

Graduate 

Enroll, 



F/E 

% Total in 
Grad. Enroll, 



U of Penn 14 642 32 
Penn State 35 268 26 
U of Pitt 39 238 26 
Camegie^Mellon 50 168 9 



19417 

33742 

22067 

5228 



6606 

5009 

7665 

1406 



34,0 

14.9 
34,7 

26.9 



Control 



PVT* 

PUB+ 

fub4 

PVT 



Bryn Mawr 


62 


101 


10 


1152 


585 




50.8 


PVT 


Lehigh 


87 


49 


3 


4982 


1830 




36,7 


PVT 


Temple 


109 


15 


3 


33284 


6197 




18,6 


PUB 


TOTALS 




1501 


111 


119872 


29298 


=t 


(24.4) 


— 


JffiANS 






15 


17124 


4185 









*3taie-A±ded 
+S tate-Related 




18 




Pennsylvania vs. the Top Fifty ; A first step is to compare 
collected information on Pennsylvania institutions with the aggre- 
gate data for the top fifty institutions. On several character- 
istics there is no real difference. Mean total enrollments are 
about the same and graduate enrollments are also very similar - 
In regard to the number of fields rated, however, there is a 
rather surprising difference. For the top fifty universities 
the mean n umb er of fields rated is twenty-six, while for Penn- 
sylvania the mean is only fifteen. 

This condition reflects more specialization within in- 
stitutions of this state than one finds in the group of top 
institutions nationally. It suggests that the Commonwealth, 
to improve the general quality of its graduate education, might 
examine the variety of graduate offerings at principal in- 
stitutions to insure a full measure of opportunity for its most 
able citizens. 



Pennsylvania vs. Selected States ; A second step involves 
the comparison of Pennsylvania data with collected information 
of other states. Information on Pennsylvania institutions is 
contained in the table above. The states in the present sample 
accounted for twenty-eight of the top fifty (56 percent). By 
individual states we find the following distribution of those 
in the top fifty versus those in the total sample: Pennsylvania' 
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4:7, Ohio— 2:4, Michigan — 2:3, New York— 7:12, Massachusetts— 
3:6, Illinois— 3: 6, and California — 7:1G. Detailed information 
for other selected states is provided in the series of tables 
which follows* 



TABLE 2 



BASIC DATA FOR 

SELECTED INSTITUTIONS IN SEVEN STATES 



Table 2B 



Ohio 



A 


B 


C 

Composite 


D 

Fields 


E 

Total 


F 

Graduate 


F/E 

% Total In 


G 


Institution 


Rank 


Score 


Rated 


Enroll* 


Enroll * 


Grad . Enroll , 


Control 


Ohio State 


28 


352 


32 


38834 


7602 


19,6 


PUB 


Case Western 


Reserve 34 


291 


21 


10927 


3606 


33,0 


PVT 


Cincinnati 


71 


78 


11 


26627 


3804 


14,3 


PUB 


Ohio U 


113 


5 


1 


16287 


1898 


11.7 


PUB 


TOTALS 




726 


65 


92675 


16910 


= (13,3) 




MEANS 






16 


23168 


4227 







Table 20 

Michigan 



A 


B 


C 


D 


E 


F 


F/E 


G 






Composite 


Fields 


Total 


Graduate 


% Total in 




Institution 


Rank 


Score 


Rated 


Enroll. 


Enroll . 


Grad* Enroll, 


Control 


Michigan 


3 


964 


36 


37283 


10328 


27.7 


PUB 


Michigan State 


26 


389 


30 


38758 


10439 


26.9 


PUB 


Wayne State 


84 


50 


9 


32370 


6939 


21.4 


PUB 


TOTALS 




1403 


75 


108411 


27706 


- (26.0) 




MEANS 






25 


36137 


9235 
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TABLE 2 (eont'd.) 



Table 2D 



New York 



A 


B 


G 


B 


E 


F 


F/E 


G 






Composite 


Fields 


Total 


Graduate 


% Total in 




Institution 


Rank 


Score 


Rated 


Enroll. 


Enroll, 


Grad. Enroll, 


Control 


Cornell 


10 


790 


33 


14102 


3870 


27,4 


PVT 


Columbia 


12 


734 


34 


17459 


6528 


37.3 


PVT 


NYU 


27 


363 


28 


34582 


14602 


42.2 


PVT 


U of Rochester 


30 


330 


24 


8423 


2043 


24.3 


PVT 


Rockefeller U 


32 


295 


10 


138 


128 


92.8 


PVT 


SUNY Buffalo 


41 


230 


24 


19113 


4781 


25.0 


PUB 


Syracuse U 


46 


192 


26 


20254 


4382 


21.6 


PVT 


Yeshiva U 


53 


145 


8 


5528 


1513 


27,4 


PVT 


Brooklyn Poly 


68 


81 


6 


5715 


2995 


52.4 


PVT 


Rennselaer 


77 


60 


8 


5144 


1355 


26.3 


PVT 


Fordham 


99 


20 


4 


10542 


1897 


18.0 


PVT 


New School 


99 


20 


4 


2609 


2125 


81,5 


PVT 


totals 




3260 


209 


143609 


46219 


(32,2) 




MEANS 






17 


11967 


3851 







Table 2E 






















Massachusetts 








A 


B 


c 


D 


E 


F 


F/E 


G 






Composite 


Fields 


Total 


Graduate 


% Total in 




Institution 


Rank 


Score 


Rated 


Enroll , 


Enroll . 


Grad. Enroll. 


Control 


Harvard 


2 


1095 


34 


19136 


6263 


32.8 


PVT 


MIT 


14 


642 


20 


7730 


3390 


43.9 


PVT 


Brandeis 


33 


294 


15 


2707 


744 


27,5 


PVT 


U of Mass 


54 


134 


17 


15202 


3467 


22.8 


PUB 


Boston U 


88 


45 


9 


23011 


5134 


22,3 


PVT 


Tufts 


93 


35 


6 


5048 


825 


16.3 


PVT 


TOTALS 




2245 


101 


72834 


19828 


(27,2) 




MEANS 






16 


12139 


3304 
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Table 2F 



Illinois 



A 


E 


C 


D 


E 


F 


F/E 


G 






Composite 


Fields 


Total 


Graduate 


% Total in 




Institution 


Rank 


Score 


Rated 


Enroll , 


Enroll . 


Grad , Enroll. 


Control 


Chicago 


7 


853 


31 


10464 


6041 


57.7 


PVT 


Illinois 


9 


819 


36 


44806 


9387 


21.0 


PUB 


Northwestern 


21 


510 


32 


17239 


4084 


23.7 


PVT 


Illinois Tech 


84 


50 


7 


8471 


2023 


23.9 


PVT 


Southern 111 


97 


25 


4 


19260 


3736 


19.4 


PUB 


Loyola (Chieago) 


109 


10 


2 


12651 


3133 


24.8 


PVT 




TOTALS 




2267 


112 


112891 


28404 


(25.2) 




MEANS 






20 


18815 


4734 







Table 2G 



California 



A 


B 


C 


D 


E 


F 


F/E 


G 






Composite 


Fields 


Total 


Graduate 


% Total in 




Institution 


Rank 


Score 


Rated 


Enroll. 


Enroll, 


Grad, Enroll. 


Control 


U, C * — Berkeley 


1 


1156 


35 


26863 


9144 


31.7 


PUB 


Stanford 


4 


945 


32 


11556 


5326 


46,1 


PVT 


U « C ,L, A, 


11 


788 


34 


29070 


8759 


30.1 


PUB 


Cal Tech 


20 


513 


16 


1520 


774 


50.9 


PVT 


U.C. — Davis 


37 


281 


16 


10161 


2367 


23,3 


PUB 


U.C.— San Diego 


40 


236 


10 


3070 


1148 


37,4 


PUB 


Southern Cal 


45 


206 


23 


18692 


6223 


33,3 


PVT 


U.C. — Riverside 


57 


122 


13 


4183 


1121 


26.6 


PUB 


Claremont Grad School 


75 


64 


7 


968 


939 


97.0 


PVT 


U.C. — San Fran Med 


76 


62 


4 


2338 


394 


16.6 


PUB 


TOTALS 




4373 


190 


110421 


36195 


(32.8) 




MEANS 






19 


11042 


3620 
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We have already noted that the policy by which institutions 
of higher learning receive funds in Pennsylvania is unique. There 
are "state-owned 11 institutions—the fourteen state colleges, 
a group of three "state-related 11 Commonwealth universities, and 
three major "state-aided" private institutions. In effect, 
the resources of the state are distributed rather than concen- 
trated in a smaller group of wholly public institutions. 

New York is the only counterpart in this tradition, although 
that state has undertaken major change toward the development 
of strong public institutions over the last decade. 

The two states are quite similar in average number of 
fields rated- This appears to confirm the idea noted above 
that when state support is extended to more institutions, more 
specialization in graduate programs develops. On the matter 
of average total enrollment there is less similarity, with New 
York institutions showing a mean of 11,967 and Pennsylvania — 
17,124. The share of the enrollment in graduate programs is 
quite different, 32 percent in New York and only 24 percent in 
Pennsylvania. Pennsylvania may have good reason to expand its 
graduate opportunity, as well as the number of programs offered. 

However, the particular policy of state funding for pri- 
vate institutions may or may not bear any relationship to the 
actual existence of quality graduate level institutions in that 
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state. For example, Massachusetts is very heavily represented 
by the private sector, yet the state allocates absolutely no 
funds for the direct operating costs of private institutions. 
Clearly then, we need to examine the broader question of the 
rated quality of public and private institutions in the selected 
states irrespective of state policy* 

Table 3 shows a summary of rated quality in the seven 
selected states aggregated by control. It is from this inf cre- 
mation that differences between the states can be seen most 
vividly. Not merely state policy but historical development 
is reflected in the particular divisions between public and 
private* On one polar extreme are New York and Massachusetts, 
which up until very recently have both placed the burden for 
graduate level work on the private institutions of the state* 

At the other extreme is Michigan which gains all of its composite 
strength from public institutions. Pennsylvania occupies middle 
ground, splitting seven institutions about as evenly as one 
can mathematically, with "three and one-half" institutions in 
each category* Ohio and Illinois fall on either side of 
Pennsylvania, with Ohio leaning toward the New York-Massaehusetts 
policy and Illinois tending toward Michigan's. California shows 
a rather balanced posture and great strength, with somewhat 
more power in the public sector. 
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^Includes two "state- related" institutions. 
* Includes one "state-aided" Institution. 



The mean scores for the rated institutions in each state 



may be taken as a rough indication of what that particular 
state is getting as a return on its total investment, public 
and private* in graduate education- Pennsylvania is clearly 
better off than Ohio but substantially below all the others. 

However desirable it might be, one can say little more on this 
matter because a number of the institutions are really national 
or international in the scope of their clientele, thus encompass- 
ing more than state interests. 

One can also see quite clearly from this table that the states 
vary in regard to their relative concentration or dispersion of 
quality graduate programs among institutions. Regarding the subject 
of dispersion, one can quickly see from Table 3 that Pennsylvania 
amasses a composite score very near Michigan’s, but does so 
through seven institutions in contrast to Michigan’s three. 
California reflects a higher composite score than New York with 
fewer institutions, while Massachusetts and Illinois have com- 
parable scores with a like number of institutions for each* 

Another way to compare Pennsylvania with other states is 
through an analysis of states with roughly equivalent popula- 
tions as seen in Table 4. Ohio and Illinois come closest to 
Pennsylvania, and one might expect them to show roughly equiva- 
lent quality levels. The three, however, show very different 
total composite scores of quality. Pennsylvania’s score is 
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TABLE 4 



SUMMED COMPOSITE SCORES IN SELECTED STATES 
AS A FUNCTION OF POPULATION IN STATE 



A 

State 


B 

Sum of 
Composite 
Scores 


c 

Population 
of State* 
(in thousands) 


B/C 

Score Points 
Per Capita 


Rank 


Pennsylvania 


1501 


11,750 


.128 


6 


Ohio 


726 


10,610 


.068 


7 


Michigan 


1403 


8,673 


.162 


5 


New York 


3260 


18,186 


.179 


4 


Massachusetts 


2245 


5,438 


.413 


1 


Illinois 


2267 


10,958 


.207 


3 


California 


4373 


19,179 


.228 


2 



*As of July 1* 1968 

SOURCE: Department of Commerce, Bureau of the Census, Statistical 

Abstract of the United States , 1970 . 

more than double that of Ohio, and Illinois' is half again higher 
than Pennsylvania's. The table presents a ratio of judged 
quality per state to population. While this measure is meant 
to assess general accessibility to quality graduate programs 
for residents of a state, it does not reflect two other impor- 
tant attributes of actual access — degree of selectivity and 
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amount of student cost. If one takes the ./♦.to of scores to 
population, it is clear that Pennsylvania ex ten ** *“o its citizens 
a rather limited opportunity for graduate study of high quality. 

Table 5 shows another aspect of general accessibility. 

The population under scrutiny here, however, is college students 
who are residents of the selected states. The first comparison 
relates total score to the number of residents who are students 
at some college in the United States. The second comparison 
is of quality scores to the number of residents who attend 
college in their home state. Table 5, may be taken 
as an indication of the quality level a state offers, first, 
to all its students, and second, to those who choose to remain 
in the state. Pennsylvania retains its relatively low posit, 
in terms of the whole student group, but it is quite similar 
to New York, We notice, however, that Pennsylvania has the 
smallest share of students remaining at home. The ratio of 
quality to students. Column B/E, looks good — comparable to 
Michigan, New York, and even California — until we remind our- 
selves that this is achieved by having more than a fifth of the 
student population go elsewhere. 

17 W, Willingham, Free-AoaeBB Higher Education (New York: 
College Entrance Examination Board, 1970). 
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Because the composite scores reflect the quality of graduate 
programs only* there is reason for looking only at the graduate 
student population. Table 6 makes the same comparisons as Table 
5 with only graduate students considered. A quick comparison 
of the percentage remaining in the home state as reflected in 
Table 5 and Table 6 reveals a little more mobility for the 
graduate students. Californians still find their opportunity 
at home as do the young people of Michigan. New York appears 
to have better holding power for its graduate students than for 
the whole range of student population. Pennsylvania still sends 
a sizeable 23 percent elsewhere and offers those who stay a 
graduate education of moderate quality. By and large the relation- 
ships remain the same among our seven sample states. 

Looking into relationships in the Roose-Andersen report 
leads one to the question of how stable the results might appear 
in light of the earlier data of the Carttcr report. To round out 
this obvious dimension of inquiry a set of composite scores for 
the 1964 report was calculated and the basic comparisons made. 

The comparison of composite scores for those in the top fifty 
in 1964 with those in the same group for 1969 showed a very 
high similarity, r » .98. Among the main subgroups reported 
in the paragraphs above“publie vs. private, selected states vs. 
top fifty“no significant differences appeared* One interesting 
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SUMMED' COMPOSITE SCORES IN SELECTED STATES 
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SOURCE: National Center for Educational Statistics,, Residence and Migration of College Students, Fall, 1968 .. 



piece of additional information did emerge when the top fifty 
for each of the two years was split in half. The top twenty- 
five In the two reports showed a correlation of ,95 by Pearson's 
method while the low twenty-five gave a value of ,77, The locus 
of change is clearly at the lower end of the scale. The strategy 
for institutional improvement when confronted with this type 
of rating system is confirmed; attempt to bring many programs 
up to a good level and hold them there. While there is little 
likelihood of a new institution reaching the top twenty-five, 
the second twenty-five is still quite acceptable company. The 
major similarities we have noted suggest that a complete repli- 
cation of this study using 1964 data would give rather similar 
results. The sone of change is the second quartile, as Table 7 
shows * 



TABLE 7 

CORRELATION BETWEEN 1969 AND 
1964 COMPOSITE SCORES 
(PEARSON 1 S METHOD) 



Top 5G—A11 Institutions .98* 

Top 50 — 

Upper 25 Only ,95* 

Lower 25 Only ■ ,77* 

Top 50- 

Private Only ,98* 

Public Only .98* 

Top 50 — 

Selected States Only .98* 

Pennsylvania — All Rated Institutions ,99* 



*Signif leant at p .05, 
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CONCLUSIONS 



By using the composite scores derived from the Roose- 
Andersen study as a reflection of the quality of graduate 
offerings in institutions, we have been able to generate some 
information and some indications that cannot be constructed 
easily by any other means. 

First , the composite scores for institutions were tested for 
relationship with several other attributes: number of fields 

rated, total enrollment, and graduate enrollment. 

1 „ There is a strong relationship between the number of 

fields rated and the composite score r = *72. This 
suggests that the avenue of institutional improvement 
lies in increasing the number of good departments in 
the graduate school rather than in creating one or 
two "star" departments. 

2 — in general, the size of the enrollment, either total \ 



or only graduate, is not significantly correlated 
with quality as measured by composite score* In 
the group of public institutions, however, there is 
an indication that larger numbers of graduate students 
do relate favorably to higher quality. 



| 




3 — Institutions which have over an extended period of 

time awarded a large number of doctorates tend to be 
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rated highly , This is attested to by the high 
correlation between number of doctorates awarded in 
the last twenty years and composite score. Greater 
doctoral production allows greater familiarity with 
an institution’s programs* 

Second y the composite scores for institutions were ag- 
gregated for the top fifty schools on the list and for seven 
representative states to give a measure of centrality. The 
collected score for Pennsylvania’s institutions was compared to 
those of the other states. Each of the six companion states 
held one or more points of similarity or contrast with Penn- 
sylvania on population, policy, or tradition. 

4 — In comparison with the top fifty institutions, Penn- 

sylvania showed fewer fields rated per institution, 
fifteen vs* twenty-six. In view of finding tfl » 
above, and in the light of what is known about the 
beneficial cross-fertilization among departments, 
Pennsylvania might well encourage the development of 
a larger number of graduate programs In its better 
institutions . 

5 — In comparison with the seven selected states, 

Pennsylvania has a relatively low proportion of its 
total student enrollment In graduate work, 24 
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percent. Only Ohio is smaller. This reinforces the 
Idea that graduate opportunity for citizens of the 
Commonwealth should be expanded. 

6 — When the scores of all institutions in a state are 

summed, we have an approximation of the quality level 
of graduate education, public and private, available 
to the people# s The Pennsylvania institutions gathered 
a total of 1501 score points, placing the state fifth 
among the seven sample states# 

7 — When this total score is related in a simple ratio 

to the factors used in our study, the position of 
Pennsylvania slips back to sixth place. For example, 
the ratio of quality points per institution puts the 
state in the sixth position* The ratio of quality 
score points to population gives a per capita value 
which also places the state sixth (Table 4) * A 
similar rank appears when the ratio of quality score 
points per student is calculated (Table 5), but 
here the unfavorable position is further intensified 
by the fact that a sizeable share of the students, 
22-23 percent, leave the state to study. 

8 ■ — A general conclusion is rather clear. Pennsylvania's 
unique policy of diversified support for higher 
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education does not, in and of itself, appear to be 
either favorable or unfavorable to the growth of 
quality in graduate study. To serve her citizens, 
Pennsylvania needs more graduate programs of good 
quality in its major institutions. Whether the 
current policy can encourage the kind of development 
needed to bring the Commonwealth up to the level of 
what might be called her "peer states" is the real 
question. And, in the light of the data presented 
here, the emphasis on more high quality graduate 
programs is no mere cry for expansion. It is the simple 
recognition of the need for an investment that has 
been too long postponed. 
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